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* NOTICES * 



JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



EFFECT OF THE INVENTION 



[Effect of the Invention] Since according to this invention the information relevant to the photographic subject which the user specified on 
the display screen of an image inputted from a video camera is retrieved and it provides for a user while a user changes the sense and the 
rate of a zoom of a video camera as explained above, even if a user knows neither the identifier of a photographic subject to search, nor its 
keyword, he can get the related information of this photographic subject only by specifying the photographic subject on an image itself 
directly. For example, many automobiles are shown to a user in the trade fair of an automobile etc., and there is a very remarkable 
advantage on multimedia information utilization centering on an image like [ if an automobile with a user is specified, while the automobile 
will zoom in / when the data is displayed and a user directs the bonnet of the automobile ] engine detailed drawing and data being displayed. 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original precisely. 
2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing I] It is the block diagram showing the configuration of the image information offer equipment which enforces the image information 
offer approach concerning the gestalt of operation of this invention. 

[Drawing 2] It is drawing for explaining the definition approach of the positional information stored in the photographic subject information 
storing means currently used for the image information offer equipment of drawing 1 . and size information. 

[Drawing 3] It is drawing for explaining the function of the coordinate transformation means of the image information offer equipment of 

drawing 1 . 

[Description of Notations] 

101 Image Information Offer Equipment 

102 Camera-Control Means 

103 Photographic Subject Information Storing Means 

104 Coordinate Transformation Means 

105 Photographic Subject Information Retrieval Means 

106 Display-Size Decision Means 

107 Photographic Subject Information Selection Means 

108 Video Camera 

109 User 

110 Display 

T Photographic subject 

A, B, C, D Rectangle surrounding T 

GThe core of T 

R Unit sphere 

X, Y, Z Axis of coordinates 

O Zero 

a, b, o, d R onto mapping of A, B, C, and D 

g R onto mapping of G 

thetag, gammag Polar coordinate of g 

H, I, J, K Graphic display screen 

M The core of H. I, J, and K 

P A user's specified location 

h, i, j, k R onto mapping of H, I, J. and K 

m R onto mapping of M 

p R onto mapping of P 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates in more detail to the image information offer approach and the equipment with which retrieve 
from a database the information relevant to the photographic subject which the user specified on the image inputted at the real time from 
the video camera, and a user provides about the image information offer approach and the equipment which provide a user with the 
information relevant to the photographic subject in the image picturized with the video camera, while a user changes the sense and the rate 
of a zoom of a video camera. 
[0002] 

[Description of the Prior Art] The sense and the rate of a zoom of a video camera are matched and memorized to each of the manual 
operation button which has more than one, and if a manual operation button is pushed by the user, the equipment to which a video camera 
is moved so that the sense and the rate of a zoom of a video camera corresponding to the manual operation button may be suited is shown 
in the report "a presetting (automatic tracking) system" of the September, 1994 issue printing of a journal "image information" (Industrial 
Development Device publication). If the identifier of the photographic subject which memorizes and corresponds to a manual operation 
button in quest of the sense and the rate of a zoom of a video camera for projecting focusing on it in this equipment for every photographic 
subject which can be caught with a video camera is given as a label of a manual operation button, unless the location where a photographic 
subject exists will change, a user can specify a photographic subject to see by the identifier, and can see that image through a video 

[0003] In addition to the sense and the rate of a zoom of a video camera, also match with a manual operation button related information, 
such as a text which described the description of a photographic subject further, and it is memorized, and it is possible to offer the related 
information according to the image of the photographic subject specified by a user. 
[0004] 

[Problem(s) to be Solved by the Invention] Although the conventional image information offer approach mentioned above could be applied 
when the user knew the identifier of an interested photographic subject and wanted to search the image and related information from the 
identifier of a photographic subject, it specified directly the photographic subject in which the user looked at the image and got interested 
on the image, and had the problem that the identifier or related information could not be searched. 

[0005] This invention was made in view of the above, and the place made into the object is to offer the image information offer approach 
and equipment which offer the information relevant to the photographic subject which the user specified directly on the image inputted at 
the real time from the video camera, while a user changes the sense and the rate of a zoom of a video camera. 
[0006] 

[Means for Solving the Problem] In order to attain the above-mentioned object, this invention according to claim 1 The information relevant 
to the photographic subject projected on the image inputted from a video camera at the real time is stored in the database. While a user is 
the image information offer approach with which identifies the photographic subject specified on said image, and a user is provided in quest 
of the information relevant to this photographic subject from a database and fixes and installs said video camera in a position The sense 
and the rate of a zoom of this video camera are changed to arbitration. The photographic subject image projected on the image inputted 
from said video camera is mapped to the unit sphere which made the installation location of a video camera the zero. When the positional 
information and size information on the unit sphere of this map are given to the information relevant to a photographic subject, and are 
memorized and a user demands modification of the sense of a video camera, or the rate of a zoom When the sense or image dilation ratio of 
a video camera is changed according to this demand and a user specifies a photographic subject on the display screen of an image inputted 
from a video camera Let it be a summary to ask for the location on said unit sphere corresponding to the specified screen location, and to 
search for the information relevant to the photographic subject currently displayed on the screen location which the user specified by 
comparing said positional information and size information which have been remembered to be this positional information searched for. 
[0007] If it is in this invention according to claim 1 , while mapping the photographic subject image in the image inputted from the video 
camera to an unit sphere, giving the positional information and size information on the unit sphere of this map to the information relevant to 
a photographic subject and memorizing them Changing the sense or image dilation ratio of a video camera by demand of a user Ask for the 
location on the unit sphere corresponding to the screen location of a photographic subject which the user specified on the display screen of 
an image inputted from a video camera, and the positional information and size information which have been remembered to be this 
positional information are compared. The information relevant to the photographic subject currently displayed on the screen location 
specified by a user is searched for. 

[0008] Moreover, the sense and the rate of a zoom of a video camera at the time of said positional information and size information which 
are given to related information about each of one or more of said related information for which it asked, and a user specifying in invention 
according to claim 1 are used for this invention according to claim 2. Only when you ask for the display size on the graphic display screen 
of the photographic subject corresponding to said related information and this display size for which it asked is within the limits of 
predetermined, let it be a summary to offer this related information to a user. 

[0009] If it is in this invention according to claim 2, it asks for the display size on the graphic display screen of the photographic subject 
corresponding to related information, and only when this display size is within the limits of predetermined, this related information is offered 

[0010] Furthermore, this invention according to claim 3 stores in the database the information relevant to the photographic subject which 
exists in the image inputted from a video camera at the real time. It is image information offer equipment which a user identifies the 
photographic subject specified on said image, and offers from a database to a user in quest of the information relevant to this photographic 
subject. A camera-control means to control the sense or the rate of a zoom of a video camera according to a demand of a user, The 
photographic subject image projected on the image inputted from said video camera is mapped to the unit sphere which made the 
installation location of a video camera the zero. A photographic subject information storing means to give the positional information and size 
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information on the unit sphere of this map to the information relevant to this photographic subject, and to memorize them, A coordinate 
transformation means to ask for the location on said unit sphere corresponding to the screen location which the user specified on the 
display screen of an image inputted from a video camera. Let it be a summary to have a photographic subject information retrieval means to 
search for the information relevant to the photographic subject currently displayed on said screen location which compared said positional 
information searched for using this coordinate transformation means, said positional information memorized by said photographic subject 
information storing means, and size information, and the user specified. 

[001 1] If it is in this invention according to claim 3, while mapping the photographic subject image in the image inputted from the video 
camera to an unit sphere, giving the positional information and size information on the unit sphere of this map to the information relevant to 
a photographic subject and memorizing them Controlling the sense or the rate of a zoom of a video camera by demand of a user Ask for the 
location on the unit sphere corresponding to the screen location of a photographic subject which the user specified on the display screen of 
an image inputted from a video camera, and the positional information and size information which have been remembered to be this 
positional information are compared. The information relevant to the photographic subject currently displayed on the screen location 
specified by a user is searched for. 

[0012] This invention according to claim 4 is set to invention according to claim 3. Furthermore, said photographic subject information 
retrieval means The sense and the rate of a zoom of a video camera at the time of said positional information and size information which 
are given to related information about each of one or more of said related information for which it asked, and a user specifying are used. 
The display size for which it asked with a display-size decision means to ask for the display size on the graphic display screen of the 
photographic subject corresponding to said related information, and this display-size decision means makes it a summary to have a 
photographic subject information selection means to choose only the related information of the photographic subject which is within the 
limits of predetermined. 

[0013] If it is in this invention according to claim 4, the display size on the graphic display screen of the photographic subject corresponding 
to related information chooses only the related information of the photographic subject which is within the limits of predetermined. 
[0014] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this invention is explained using a drawing. 
[001 5] Drawing 1 is the block diagram showing the configuration of the image information offer equipment which enforces the image 
information offer approach concerning the gestalt of operation of this invention, this drawing — setting — 101 — image information offer 
equipment and 102 — a camera-control means and 103 — a photographic subject information storing means and 104 — for a display-size 
decision means and 107, as for a video camera and 109, a photographic subject information selection means and 108 are [ a coordinate 
transformation means and 105 / a photographic subject information retrieval means and 106 / a user and 110] displays. 
[0016] The video camera 108 is attached in the four directions of vertical and horizontal at the pivotable universal head, and has 
composition which it is remotely operated with the camera-control means 102, and can change the sense of a camera. Furthermore, the 
rate of a zoom of a lens is also adjustable and this can also operate it by remote control with the camera-control means 102. However, 
suppose that the installation location of a video camera 108 is being fixed. The image inputted from a video camera 108 is displayed on up 
to a display 110. 

[001 7] The camera-control means 1 02 controls a video camera 1 08 to receive the demand for changing the sense or the rate of a zoom of 
a video camera 108 from a user 109, and to suit the demanded sense or the rate of a zoom. For example, a video camera 108 can be moved 
rightward by giving the demand for rotating a video camera 108 rightward to the camera-control means 102 and desired space can be 
projected [ of the space in the image as which the user 109 is displayed on the display 110 now ] to see right-hand side to a slight degree. 
For example, when a user 109 wants to enlarge the photographic subject in the image displayed on the display 110 now to a slight degree, 
and to see it, by giving the camera-control means 102, the rate of a zoom of a video camera 108 can be expanded to the size of a request 
of raising and a photographic subject, and the demand for making the rate of a zoom of a video camera 108 increase can be seen. In 
addition, as technique for a user 109 to demand modification of the sense of a video camera 108, or the rate of a zoom, how to display a 
control panel on a display 110, the method of forming the controller for remote control, etc. can be considered. 

[0018] The photographic subject information storing means 103 has memorized related information, such as a text which described the 
description of each photographic subject about the photographic subject from which an existence location does not change in the actual 
world. When the photographic subject turns a video camera 108 in which direction, the positional information and size information which 
show whether it appears in an image are given to the related information of each photographic subject. Positional information and size 
information can be defined as the installation location of a video camera 108 is shown in drawing 2 , using the unit sphere made into the 
zero. In drawing 2 , the video camera is fixed and installed in Zero O. The point of the arbitration on an unit sphere R can express the 
location by the polar coordinate (theta, gamma) which used the angle of rotation theta from the X-axis for example, on XY flat surface to Y 
shaft orientations, and the angle of rotation gamma from XY flat surface to Z shaft orientations. Therefore, the positional information and 
size information on a photographic subject T in drawing 2 For example, the points A, B, C, and D of the four comers of the rectangle 
surrounding the central point G of a photographic subject T and a photographic subject T are mapped to up to an unit sphere, respectively, 
a map — a point — g — a — b — c — d — a location — expressing — a polar coordinate — a group — {(thetag and gammag) — (— 
theta — a — gamma — a — ) — (— theta — b — gamma — b — ) — (— theta — c — gamma — c — ) — ) (thetad and gammad) — 
****** _ it can express . 

[0019] The coordinate transformation means 104 will search for the point p on said unit sphere R corresponding to the specified screen 
location P as follows, for example, if a user 109 specifies a specific photographic subject on the image currently displayed on the display 
110 using pointing equipment like a mouse. As shown in the example of drawing 3 , in the condition of having turned the video camera 108 in 
the specific direction, the central point M of the image displayed on a display 110 is always matched with the point m of a meaning on said 
unit sphere R. On the other hand, the points h, i, j, and k on the unit sphere R corresponding to the points H, I, J, and K of the four comers 
of a graphic display screen change with the rates of a zoom of a video camera 108. Although the rate of a zoom of a video camera 108 and 
the relation of Points h, i, j, and k change with video cameras 108 to be used, this relation can be defined using a formula or a conversion 
table every video camera 108. The coordinate transformation means 104 will ask for the sense and the rate of a zoom of a video camera 
108 at the event using the camera-control means 102, if a user 109 specifies a photographic subject on a graphic display screen. 
[0020] Next, the coordinate of the point m on the unit sphere R corresponding to the central point M of the image by which it is indicated 
by current from the sense of the video camera 1 08 for which it asked is searched for. Next, the coordinate of the points h, i, j, and k at the 
event is searched for using the rate of a zoom of the video camera 108 for which it asked, and the coordinate of Point m. And the 
coordinate of the point p on the unit sphere R corresponding to the screen location P specified by a user 109 is computed from the relative 
physical relationship of the coordinate of Points h, i, j, and k and Points H, I, J, and K which were searched for, and Point P. The coordinate 
transformation means 104 gives the coordinate of the point p searched for to the photographic subject information retrieval means 105. 
[0021] The photographic subject information retrieval means 105 reads the related information of the photographic subject stored in the 
photographic subject information storing means 103, compares the coordinate of the point p given from said positional information and size 
information which are given to them, and the coordinate transformation means 104, and searches the related information of the 
photographic subject which exists on the graphic display screen specified by a user 109. If the positional information and size information on 
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the unit sphere R given to photographic subject information include the coordinate of the four comers a, b, c, and d of the rectangle which 
surrounds a photographic subject image as mentioned above, the coordinate of Point p can sort out the related information which 
corresponds with confirming whether it is contained in the rectangle surrounding each photographic subject. The related information 
obtained by retrieval is not necessarily one or less. For example, while regarding the whole "automobile" as a photographic subject and 
storing related information called the identifier and model in the photographic subject information storing means 103 If it regards as one 
photographic subject with which the "tire" of the automobile which is elegance a part became independent and related information called 
the manufacturer and size is also simultaneously stored in the photographic subject information storing means 103 When a user 109 
specifies the "tire" of a "automobile" on a graphic display screen, the related information obtained using the photographic subject^ 
information retrieval means 105 turns into both the information relevant to a "automobile", and the information relevant to a tire. 
[0022] furthermore, as information relevant to a photographic subject in detail It adds to the ability of numeric data information about this 
image information, such as property data and structure data, etc. to be offered as related information by using not only the image 
information of a photographic subject but this image information as a key. Only not only in the information on the photographic subject 
which has appeared directly on the display screen as information relevant to this photographic subject as mentioned above usually, although 
it is thought that it exists, when the bonnet of the indirect thing which does not look direct, for example, an automobile, is displayed An 
"engine" etc. can be offered as related information, for example, and not only that image information but the information on property data, 
such as that engine performance, structure, etc., etc. can be offered [ which it is considered to usually exist in this bonnet bottom ] as 
related information of this engine. 

[0023] Next, the photographic subject information retrieval means 1 05 asks for the screen-display size in the event of the photographic 
subject corresponding to each related information as follows about one or more related information obtained by retrieval, using the display- 
size decision means 106. First, it asks for the sense and the rate of a zoom of a video camera 108 at the event using the camera-control 
means 102. Next, the points h, i, j, and k on the unit sphere R corresponding to the points H, [, J, and K of the four corners of a graphic 
display screen are searched for by the same technique as the coordinate transformation means 104. Next, using said positional information 
and size information which are given to the coordinate and each related information of the points h, i, j, and k searched for, each 
photographic subject image computes the field of what percent of a graphic display screen is occupied, and makes it the screen-display 
size of each photographic subject. 

[0024] Furthermore, about one or more related information obtained by retrieval, the photographic subject information retrieval means 105 
chooses what fulfills predetermined conditions using the photographic subject information selection means 107, and said screen-display size 
for which it asked with the display-size decision means 106 displays only the selected related information on a display 110, and offers it to 
a user 109. As for the related information searched using the photographic subject information retrieval means 105, a user 109 does not^ 
necessarily ask for the all. For example, although two. the information relevant to a "automobile" and the information relevant to a "tire", 
will be obtained in above-mentioned "automobile" and the example of a "tire" using the photographic subject information retrieval means 
105 if a user 109 specifies a "tire" on a graphic display screen If the user 109 specified the "tire" in the condition of having zoomed in, at 
this time, the suitable related information which should be offered to a user 109 is related with a "tire." On the other hand, the whole 
"automobile" is reflected in the screen, and if a "tire" is in the condition currently displayed so small that it is not set as the object of 
interest, it can be judged that a user's 109 interest is the information relevant to a "automobile." The photographic subject information 
selection means 107 performs selection of related information which was described above based on the screen-display size of the 
photographic subject searched for with the display-size decision means 106. If this screen-display size is expressed as a rate of a 
photographic subject image of occupying a graphic display screen, range conditions which it says, for example are 30% or more and 80% or 
less will be established, and only the related information to which the display size which fulfills this condition is given will be chosen. 
Although the example which chooses related information only using the rate of a photographic subject image of occupying a graphic display 
screen was shown here, it is also possible to realize more exact selection by in which location of a graphic display screen the core of a 
photographic subject exists and using information further. 
[0025] 

[Effect of the Invention] Since according to this invention the information relevant to the photographic subject which the user specified on 
the display screen of an image inputted from a video camera is retrieved and it provides for a user while a user changes the sense and the 
rate of a zoom of a video camera as explained above, even if a user knows neither the identifier of a photographic subject to search, nor its 
keyword, he can get the related information of this photographic subject only by specifying the photographic subject on an image itself 
directly. For example, many automobiles are shown to a user in the trade fair of an automobile etc., and there is a very remarkable 
advantage on multimedia information utilization centering on an image like [ if an automobile with a user is specified, while the automobile 
will zoom in / when the data is displayed and a user directs the bonnet of the automobile ] engine detailed drawing and data being displayed. 
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CLAIMS 
[Claim(s)] 

[Claim 1] The information relevant to the photographic subject projected on the image inputted from a video camera at the real time is 
stored in the database. While a user is the image information offer approach with which identifies the photographic subject specified on said 
image, and a user is provided in quest of the information relevant to this photographic subject from a database and fixes and installs said 
video camera in a position The sense and the rate of a zoom of this video camera are changed to arbitration. The photographic subject 
image projected on the image inputted from said video camera is mapped to the unit sphere which made the installation location of a video 
camera the zero. When the positional information and size information on the unit sphere of this map are given to the information relevant 
to a photographic subject, and are memorized and a user demands modification of the sense of a video camera, or the rate of a zoom When 
the sense or image dilation ratio of a video camera is changed according to this demand and a user specifies a photographic subject on the 
display screen of an image inputted from a video camera The image information offer approach characterized by asking for the location on 
said unit sphere corresponding to the specified screen location, and searching for the information relevant to the photographic subject 
currently displayed on the screen location which the user specified by comparing said positional information and size information which have 
been remembered to be this positional information searched for. 

[Claim 2] The image information offer approach according to claim 1 characterized by offering this related information to a user only when it 
asks for the display size on the graphic display screen of the photographic subject corresponding to said related information and this display 
size for which it asked is within the limits of predetermined using the sense and the rate of a zoom of a video camera at the time of said 
positional information and size information which are given to related information about each of one or more of said related information for 
which it asked, and a user specifying. 

[Claim 3] The information relevant to the photographic subject which exists in the image inputted from a video camera at the real time is 
stored in the database. It is image information offer equipment which a user identifies the photographic subject specified on said image, and 
offers from a database to a user in quest of the information relevant to this photographic subject. A camera-control means to control the 
sense or the rate of a zoom of a video camera according to a demand of a user, The photographic subject image projected on the image 
inputted from said video camera is mapped to the unit sphere which made the installation location of a video camera the zero. A 
photographic subject information storing means to give the positional information and size information on the unit sphere of this map to the 
information relevant to this photographic subject, and to memorize them, A coordinate transformation means to ask for the location on said 
unit sphere corresponding to the screen location which the user specified on the display screen of an image inputted from a video camera, 
Said positional information searched for using this coordinate transformation means, said positional information memorized by said 
photographic subject information storing means, and size information are compared. Image information offer equipment characterized by 
having a photographic subject information retrieval means to search for the information relevant to the photographic subject currently 
displayed on said screen location specified by a user. 

[Claim 4] The sense and the rate of a zoom of a video camera at the time of said positional information and size information which are 
given to related information about each of one or more of said related information for which it asked, and a user specifying are used for said 
photographic subject information retrieval means. A display-size decision means to ask for the display size on the graphic display screen of 
the photographic subject corresponding to said related information. Image information offer equipment according to claim 3 characterized by 
having a photographic subject information selection means to choose only the related information of the photographic subject which has the 
display size for which it asked with this display-size decision means within the limits of predetermined. 
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MEANS 



[Means for Solving the Problem] In order to attain the above-mentioned object, this invention according to claim 1 The information relevant 
to the photographic subject projected on the image inputted from a video camera at the real time is stored in the database. While a user is 
the image information offer approach with which identifies the photographic subject specified on said image, and a user is provided in quest 
of the information relevant to this photographic subject from a database and fixes and installs said video camera in a position The sense 
and the rate of a zoom of this video camera are changed to arbitration. The photographic subject image projected on the image inputted 
from said video camera is mapped to the unit sphere which made the installation location of a video camera the zero. When the positional 
information and size information on the unit sphere of this map are given to the information relevant to a photographic subject, and are 
memorized and a user demands modification of the sense of a video camera, or the rate of a zoom When the sense or image dilation ratio of 
a video camera is changed according to this demand and a user specifies a photographic subject on the display screen of an image inputted 
from a video camera Let it be a summary to ask for the location on said unit sphere corresponding to the specified screen location, and to 
search for the information relevant to the photographic subject currently displayed on the screen location which the user specified by 
comparing said positional information and size information which have been remembered to be this positional information searched for. 
[0007] If it is in this invention according to claim 1 , while mapping the photographic subject image in the image inputted from the video 
camera to an unit sphere, giving the positional information and size information on the unit sphere of this map to the information relevant to 
a photographic subject and memorizing them Changing the sense or image dilation ratio of a video camera by demand of a user Ask for the 
location on the unit sphere corresponding to the screen location of a photographic subject which the user specified on the display screen of 
an image inputted from a video camera, and the positional information and size information which have been remembered to be this 
positional information are compared. The information relevant to the photographic subject currently displayed on the screen location 
specified by a user is searched for. 

[0008] Moreover, the sense and the rate of a zoom of a video camera at the time of said positional information and size information which 
are given to related information about each of one or more of said related information for which it asked, and a user specifying in invention 
according to claim 1 are used for this invention according to claim 2. Only when you ask for the display size on the graphic display screen 
of the photographic subject corresponding to said related information and this display size for which it asked is within the limits of 
predetermined, let it be a summary to offer this related information to a user. 

[0009] If it is in this invention according to claim 2, it asks for the display size on the graphic display screen of the photographic subject 
corresponding to related information, and only when this display size is within the limits of predetermined, this related information is offered 

[0010] Furthermore, this invention according to claim 3 stores in the database the information relevant to the photographic subject which 
exists in the image inputted from a video camera at the real time. It is image information offer equipment which a user identifies the 
photographic subject specified on said image, and offers from a database to a user in quest of the information relevant to this photographic 
subject A camera-control means to control the sense or the rate of a zoom of a video camera according to a demand of a user, The 
photographic subject image projected on the image inputted from said video camera is mapped to the unit sphere which made the 
installation location of a video camera the zero. A photographic subject information storing means to give the positional information and size 
information on the unit sphere of this map to the information relevant to this photographic subject, and to memorize them, A coordinate 
transformation means to ask for the location on said unit sphere corresponding to the screen location which the user specified on the 
display screen of an image inputted from a video camera, Let it be a summary to have a photographic subject information retrieval means to 
search for the information relevant to the photographic subject currently displayed on said screen location which compared said positional 
information searched for using this coordinate transformation means, said positional information memorized by said photographic subject 
information storing means, and size information, and the user specified. 

[001 1] If it is in this invention according to claim 3, while mapping the photographic subject image in the image inputted from the video 
camera to an unit sphere, giving the positional information and size information on the unit sphere of this map to the information relevant to 
a photographic subject and memorizing them Controlling the sense or the rate of a zoom of a video camera by demand of a user Ask for the 
location on the unit sphere corresponding to the screen location of a photographic subject which the user specified on the display screen of 
an image inputted from a video camera, and the positional information and size information which have been remembered to be this 
positional information are compared. The information relevant to the photographic subject currently displayed on the screen location 
specified by a user is searched for. 

[0012] This invention according to claim 4 is set to invention according to claim 3. Furthermore, said photographic subject information 
retrieval means The sense and the rate of a zoom of a video camera at the time of said positional information and size information which 
are given to related information about each of one or more of said related information for which it asked, and a user specifying are used. 
The display size for which it asked with a display-size decision means to ask for the display size on the graphic display screen of the 
photographic subject corresponding to said related information, and this display-size decision means makes it a summary to have a 
photographic subject information selection means to choose only the related information of the photographic subject which is within the 
limits of predetermined. 

[0013] If it is in this invention according to claim 4, the display size on the graphic display screen of the photographic subject corresponding 
to related information chooses only the related information of the photographic subject which is within the limits of predetermined. 
[0014] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this invention is explained using a drawing. 
[0015] Drawing i is the block diagram showing the configuration of the image information offer equipment which enforces the image 
information offer approach concerning the gestalt of operation of this invention, this drawing — setting — 101 — image information offer 
equipment and 102 — a camera-control means and 103 — a photographic subject information storing means and 104 — for a display-size 
decision means and 107, as for a video camera and 109, a photographic subject information selection means and 108 are [ a coordinate 
transformation means and 105 / a photographic subject information retrieval means and 106 / a user and 110 ] displays. 
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[0016] The video camera 108 is attached in the four directions of vertical and horizontal at the pivotable universal head, and has 
composition which it is remotely operated with the camera-control means 102, and can change the sense of a camera. Furthermore, the 
rate of a zoom of a lens is also adjustable and this can also operate it by remote control with the camera-control means 102. However, 
suppose that the installation location of a video camera 108 is being fixed. The image inputted from a video camera 108 is displayed on up 
to a display 110. 

[0017] The camera-control means 102 controls a video camera 108 to receive the demand for changing the sense < 
a video camera 108 from a user 109, and to suit the demanded sense or the rate of a zoom. For example, a video camera 108 ca 
rightward by giving the demand for rotating a video camera 108 rightward to the camera-control means 102 and desired space can be 
projected [ of the space in the image as which the user 1 09 is displayed on the display 1 1 0 now ] to see right-hand side to a slight degree. 
For example, when a user 109 wants to enlarge the photographic subject in the image displayed on the display 1 10 now to a slight degree, 
and to see it, by giving the camera-control means 102, the rate of a zoom of a video camera 108 can be expanded to the size of a request 
of raising and a photographic subject, and the demand for making the rate of a zoom of a video camera 108 increase can be seen. In 
addition, as technique for a user 109 to demand modification of the sense of a video camera 108, or the rate of a zoom, how to display a 
control panel on a display 1 1 0, the method of forming the controller for remote control, etc. can be considered. 

[0018] The photographic subject information storing means 103 has memorized related information, such as a text which described the 
description of each photographic subject about the photographic subject from which an existence location does not change in the actual 
world. When the photographic subject turns a video camera 108 in which direction, the positional information and size information which 
show whether it appears in an image are given to the related information of each photographic subject Positional information and size 
information can be defined as the installation location of a video camera 108 is shown in drawing. 2 , using the unit sphere made into the 
zero. In drawing 2 , the video camera is fixed and installed in Zero O. The point of the arbitration on an unit sphere R can express the 
location by the polar coordinate (theta, gamma) which used the angle of rotation theta from the X-axis for example, on XY flat surface to Y 
shaft orientations, and the angle of rotation gamma from XY flat surface to Z shaft orientations. Therefore, the positional information and 
size information on a photographic subject T in dr awing 2 For example, the points A, B, C, and D of the four corners of the rectangle 
surrounding the central point G of a photographic subject T and a photographic subject T are mapped to up to an unit sphere, respectively, 
a map — a point — g — a — b — c — d — a location — expressing — a polar coordinate — a group — {(thetag and gammag) — (— 
theta — a — gamma — a —) — (— theta — b — gamma — b — ) — (— theta — c — gamma — c —) — } (thetad and gammad) — 

[0019] The coordinate transformation means 104 will search for the point p on said unit sphere R corresponding to the specified screen 
location P as follows, for example, if a user 109 specifies a specific photographic subject on the image currently displayed on the display 
1 10 using pointing equipment like a mouse. As shown in the example of drawing 3 , in the condition of having turned the video camera 108 in 
the specific direction, the central point M of the image displayed on a display 1 1 0 is always matched with the point m of a meaning on said 
unit sphere R. On the other hand, the points h, i, j, and k on the unit sphere R corresponding to the points H, I, J, and K of the four coiners 
of a graphic display screen change with the rates of a zoom of a video camera 108. Although the rate of a zoom of a video camera 108 and 
the relation of Points h, i, j, and k change with video cameras 108 to be used, this relation can be defined using a formula or a conversion 
table every video camera 108. The coordinate transformation means 104 will ask for the sense and the rate of a zoom of a video camera 
108 at the event using the camera-control means 102, if a user 109 specifies a photographic subject on a graphic display screen. 
[0020] Next, the coordinate of the point m on the unit sphere R corresponding to the central point M of the image by which it is indicated 
by current from the sense of the video camera 108 for which it asked is searched for. Next, the coordinate of the points h, i,j, and k at the 
event is searched for using the rate of a zoom of the video camera 108 for which it asked, and the coordinate of Point m. And the 
coordinate of the point p on the unit sphere R corresponding to the screen location P specified by a user 109 is computed from the relative 
physical relationship of the coordinate of Points h, i, j, and k and Points H, I, J, and K which were searched for, and Point P. The coordinate 
transformation means 104 gives the coordinate of the point p searched for to the photographic subject information retrieval means 105. 
[0021] The photographic subject information retrieval means 105 reads the related information of the photographic subject stored in the 
photographic subject information storing means 103, compares the coordinate of the point p given from said positional information and size 
information which are given to them, and the coordinate transformation means 104, and searches the related information of the 
photographic subject which exists on the graphic display screen specified by a user 109. If the positional information and size information on 
the unit sphere R given to photographic subject information include the coordinate of the four corners a, b, c, and d of the rectangle which 
surrounds a photographic subject image as mentioned above, the coordinate of Point p can sort out the related information which 
corresponds with confirming whether it is contained in the rectangle surrounding each photographic subject. The related information 
obtained by retrieval is not necessarily one or less. For example, while regarding the whole "automobile" as a photographic subject and 
storing related information called the identifier and model in the photographic subject information storing means 103 If it regards as one 
photographic subject with which the "tire" of the automobile which is elegance a part became independent and related information called 
the manufacturer and size is also simultaneously stored in the photographic subject information storing means 103 When a user 109 
specifies the "tire" of a "automobile" on a graphic display screen, the related information obtained using the photographic subject^ 
information retrieval means 105 turns into both the information relevant to a "automobile", and the information relevant to a "tire. 
[0022] furthermore, as information relevant to a photographic subject, in detail It adds to the ability of numeric data information about this 
image information, such as property data and structure data, etc. to be offered as related information by using not only the image 
information of a photographic subject but this image information as a key. Only not only in the information on the photographic subject 
which has appeared directly on the display screen as information relevant to this photographic subject as mentioned above usually, although 
it is thought that it exists, when the bonnet of the indirect thing which does not look direct, for example, an automobile, is displayed An 
"engine" etc. can be offered as related information, for example, and not only that image information but the information on property data, 
such as that engine performance, structure, etc., etc. can be offered [ which it is considered to usually exist in this bonnet bottom ] as 
related information of this engine. 

[0023] Next, the photographic subject information retrieval means 105 asks for the screen-display size in the event of the photographic 
subject corresponding to each related information as follows about one or more related information obtained by retrieval, using the display- 
size decision means 106. First, it asks for the sense and the rate of a zoom of a video camera 108 at the event using the camera-control 
means 102. Next, the points h, i, j, and k on the unit sphere R corresponding to the points H, I, J, and K of the four comers of a graphic 
display screen are searched for by the same technique as the coordinate transformation means 104. Next, using said positional information 
and size information which are given to the coordinate and each related information of the points h, i, j, and k searched for, each 
photographic subject image computes the field of what percent of a graphic display screen is occupied, and makes it the screen-display 
size of each photographic subject 

[0024] Furthermore, about one or more related information obtained by retrieval, the photographic subject information retrieval means 1Ui> 
chooses what fulfills predetermined conditions using the photographic subject information selection means 107, and said screen-display size 
for which it asked with the display-size decision means 106 displays only the selected related information on a display 110, and offers it to 
a user 109. As for the related information searched using the photographic subject information retrieval means 105, a user 109 does not 
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necessarily ask for the all. For example, although two, the information relevant to a "automobile" and the information relevant to a "tire , 
will be obtained in above-mentioned "automobile" and the example of a "tire" using the photographic subject information retrieval means 
105 if a user 109 specifies a "tire" on a graphic display screen If the user 109 specified the "tire" in the condition of having zoomed in, at 
this time, the suitable related information which should be offered to a user 109 is related with a "tire." On the other hand, the whole 
"automobile" is reflected in the screen, and if a "tire" is in the condition currently displayed so small that it is not set as the object of 
interest it can be judged that a user's 109 interest is the information relevant to a "automobile." The photographic subject information 
selection means 107 performs selection of related information which was described above based on the screen-display size of the 
photographic subject searched for with the display-size decision means 106. If this screen-display size is expressed as a rate of a 
photographic subject image of occupying a graphic display screen, range conditions which it says, for example are 30% or more and 80% or 
less will be established, and only the related information to which the display size which fulfills this condition is given will be chosen. 
Although the example which chooses related information only using the rate of a photographic subject image of occupying a graphic display 
screen was shown here, it is also possible to realize more exact selection by in which location of a graphic display screen the core of a 
photographic subject exists and using information further. 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the original precisely. 
2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



PRIOR ART 



[Description of the Prior Art] The sense and the rate of a zoom of a video camera are matched and memorized to each of the manual 
operation button which has more than one, and if a manual operation button is pushed by the user, the equipment to which a video camera 
is moved so that the sense and the rate of a zoom of a video camera corresponding to the manual operation button may be suited is shown 
in the report "a presetting (automatic tracking) system" of the September, 1994 issue printing of a journal "image information" (Industrial 
Development Device publication). If the identifier of the photographic subject which memorizes and corresponds to a manual operation 
button in quest of the sense and the rate of a zoom of a video camera for projecting focusing on it in this equipment for every photographic 
subject which can be caught with a video camera is given as a label of a manual operation button, unless the location where a photographic 
subject exists will change, a user can specify a photographic subject to see by the identifier, and can see that image through a video 
camera. 

[0003] In addition to the sense and the rate of a zoom of a video camera, also match with a manual operation button related information, 
such as a text which described the description of a photographic subject further, and it is memorized, and it is possible to offer the related 
information according to the image of the photographic subject specified by a user. 



[Translation done.] 
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* NOTICES * 



JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



TECHNICAL FIELD 



[Field of the Invention] This invention relates in more detail to the image information offer approach and the equipment with which retrieve 
from a database the information relevant to the photographic subject which the user specified on the image inputted at the real time from 
the video camera, and a user provides about the image information offer approach and the equipment which provide a user with the 
information relevant to the photographic subject in the image picturized with the video camera, while a user changes the sense and the rate 
of a zoom of a video camera. 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



TECHNICAL PROBLEM 



[Problem(s) to be Solved by the Invention] Although the conventional image information offer approach mentioned above could be applied 
when the user knew the identifier of an interested photographic subject and wanted to search the image and related information from the 
identifier of a photographic subject, it specified directly the photographic subject in which the user looked at the image and got interested 
on the image, and had the problem that the identifier or related information could not be searched. 

[0005] This invention was made in view of the above, and the place made into the object is to offer the image information offer approach 
and equipment which offer the information relevant to the photographic subject which the user specified directly on the image inputted at 
the real time from the video camera, while a user changes the sense and the rate of a zoom of a video camera. 



[Translation done.] 
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